An Adaptive Dynamic Programming Algorithm for a Stochastic Multiproduct Batch Dispatch Problem

We address the problem of dispatching a vehicle with different product classes. There is a common dispatch cost, but holding costs that vary by product class. The problem exhibits multidimensional state, outcome and action spaces, and as a result is computationally intractable using either discrete dynamic programming methods, or even as a deterministic integer program. We prove a key structural property for the decision function, and exploit this property in the development of continuous value function approximations that form the basis of an approximate dispatch rule. Comparisons on single product-class problems, where optimal solutions are available, demonstrate solutions that are within a few percent of optimal. The algorithm is then applied to a problem with 100 product classes, and comparisons against a carefully tuned myopic heuristic demonstrate significant improvements. The multiproduct batch dispatch problem consists of different types of products arriving at a dispatch station in discrete time intervals over a finite horizon waiting to be dispatched by a finite capacity vehicle. The basic decision is whether or not a batch of products should be dispatched and, in the case that the vehicle is dispatched, determining how many products to dispatch from each type. We assume that the arrival process is nonstationary and stochastic. There is a fixed cost for each vehicle dispatch but there is a different holding cost for each product type, reflecting differences in the values of product types. The single-product batch dispatch problem can be solved optimally using classical discrete dynamic programming techniques. However, these methods cannot be used in the multiproduct case since the state, outcome and action spaces all become multidimensional. In this paper we use adaptive sampling techniques to produce continuous value function approximations. We prove that the optimal solution has a particular structure, and we exploit this structure in our algorithm. These strategies are then used to produce approximate decision functions (policies) which are scalable to problems with a very large number of product classes. The single link dispatching problem was originally proposed by Kosten (1967) (see Medhi (1984), Kosten (1973)), who considered the case of a custodian dispatching trucks whenever the number of customers waiting to be served exceeds a threshold. Deb & Serfozo (1973) show that the optimal decision rule is monotone and has a control limit structure. Papadaki & Powell (2002) considered the single link dispatching problem as a batch service queue with homogeneous customers. They proved monotonicity of the optimal service rules in a finite horizon setting. Weiss & Pliska (1976) considered the case where the waiting cost per customer is a function h(w) if a customer has been waiting time w. They show that the optimal service policy is to send the vehicle if the server is available and the marginal waiting cost is at least as large as the optimal long run average cost. This is termed a derivative policy, in contrast to the control limit policy proved by Deb and Serfozo. We use both of these results in this paper. This basic model is generalized in Deb (1978a) to include switching costs, and has been applied to the study of a two-terminal shuttle system where one or two vehicles cycle between a pair of terminals Ignall & Kolesar (1972); Barnett (1973); Ignall & Kolesar (1974); Deb (1978b); Weiss (1981); Deb & Schmidt (1987). Once the structure of an optimal policy is known, the primary problem is one of determining the expected costs for a given control strategy, and then using this function to find the optimal control strategy. Since the original paper by Bailey (1954), there has developed an extensive literature on statistics for steady state bulk queues. Several authors have suggested different types of control strategies, motivated by different cost functions than those